Optimization Of Loss Function Research Articles

Non-Differentiable Loss Function Optimization and Interaction Effect Discovery in Insurance Pricing Using the Genetic Algorithm

Non-Differentiable Loss Function Optimization and Interaction Effect Discovery in Insurance Pricing Using the Genetic Algorithm

SLKIR: A framework for extracting key information from air traffic control instructions Using small sample learning

In air traffic control (ATC), Key Information Recognition (KIR) of ATC instructions plays a pivotal role in automation. The field's specialized nature has led to a scarcity of related research and a gap with the industry's cutting-edge developments. Addressing this, an innovative end-to-end deep learning framework, Small Sample Learning for Key Information Recognition (SLKIR), is introduced for enhancing KIR in ATC instructions. SLKIR incorporates a novel Multi-Head Local Lexical Association Attention (MHLA) mechanism, specifically designed to enhance accuracy in identifying boundary words of key information by capturing their latent representations. Furthermore, the framework includes a task focused on prompt, aiming to bolster the semantic comprehension of ATC instructions within the core network. To overcome the challenges posed by category imbalance in boundary word and prompt discrimination tasks, tailored loss function optimization strategies are implemented, effectively expediting the learning process and boosting recognition accuracy. The framework's efficacy and adaptability are demonstrated through experiments on two distinct ATC instruction datasets. Notably, SLKIR outperforms the leading baseline model, W2NER, achieving a 3.65% increase in F1 score on the commercial flight dataset and a 12.8% increase on the training flight dataset. This study is the first of its kind to apply small-sample learning in KIR for ATC and the source code of SLKIR will be available at: https://github.com/PANPANKK/ATC_KIR.

Robust hyperspectral image classification using generative adversarial networks

This paper introduces Sill-Rgan, a novel Generative Adversarial Network (GAN) designed to improve hyperspectral image (HSI) classification under varying lighting conditions. Sill-Rgan uniquely maps different light condition domains, enhancing sample classification robustness and generating new virtual samples. Addressing challenges like high spectral dimensionality and noise in HSI classification, our approach utilizes a deep proxy-based learning framework. It integrates and improves advanced GAN models and multitask networks for optimal training stability and loss function optimization. The model's mapping network is adept at generating domain-specific latent codes, enabling the transformation of original hyperspectral data into enhanced versions. Extensive experiments conducted on hyperspectral datasets of agricultural products under diverse indoor and outdoor lighting conditions confirm the effectiveness of Sill-Rgan. The results highlight the model's adaptability in both supervised and semi-supervised learning scenarios, yielding exceptional classification accuracy and enhanced data quality. The versatile potential of Sill-Rgan extends its applicability to a broad range of spectral data classifications, underlining its significant contribution to hyperspectral imaging. This advancement opens new avenues in machine vision systems, particularly in scenarios with dynamic lighting challenges.

An algorithm for power transmission line fault detection based on improved YOLOv4 model

In response to the escalating demand for real-time and accurate fault detection in power transmission lines, this paper undertook an optimization of the existing YOLOv4 network. This involved the substitution of the main feature extraction network within the original YOLOv4 model with a lighter EfficientNet network. Additionally, the inclusion of Grouped Convolution modules in the feature pyramid structure replaced conventional convolution operations. The resulting model not only reduced model parameters but also effectively ensured detection accuracy. Moreover, in enhancing the model's reliability, data augmentation techniques were employed to bolster the robustness of the power transmission line fault detection algorithm. This optimization further utilized the DIoU loss function to stabilize target box regression. Comparative experiments demonstrated the improved YOLOv4 model's superior performance in terms of loss function optimization while significantly enhancing detection speed under equivalent configurations. The parameter capacity was reduced by 81%, totaling merely 43.65 million, while the frame rate surged by 85% to achieve 24 frames per second. These experimental findings validate the effectiveness of the algorithm.

Real-time reliable semantic segmentation of thyroid nodules in ultrasound images

Objective. Low efficiency in medical image segmentation is a common issue that limits computer-aided diagnosis development. Due to the varying positions and sizes of nodules, it is not easy to accurately segment ultrasound images. This study aims to propose a segmentation model that maintains high efficiency while improving accuracy. Approach. We propose a novel layer that integrates the advantages of dense connectivity, dilated convolution, and factorized filters to maintain excellent efficiency while improving accuracy. Dense connectivity optimizes feature reuse, dilated convolution redesigns layers, and factorized convolution improves efficiency. Moreover, we propose a loss function optimization method from a pixel perspective to increase the network's accuracy further. Main results. Experiments on the Thyroid dataset show that our method achieves 81.70% intersection-over-union (IoU), 90.50% true positive rate (TPR), and 0.25% false positive rate (FPR). In terms of accuracy, our method outperforms the state-of-the-art methods, with twice faster inference and nearly 400 times fewer parameters. Meanwhile, in a test on an External Thyroid dataset, our method achieves 77.03% IoU, 82.10% TPR, and 0.16% FPR, demonstrating our proposed model's robustness. Significance. We propose a real-time semantic segmentation architecture for thyroid nodule segmentation in ultrasound images called fully convolution dense dilated network (FCDDN). Our method runs fast with a few parameters and is suitable for medical devices requiring real-time segmentation.

Financial Risk Prediction Model in the Context of Big Data - Corporate Financial Risk Control Based on LSTM Deep Neural Networks

Abstract This paper is based on the use of recurrent neural networks and LSTM deep neural networks to obtain the financial risk prediction feature sequence in the context of big data. The financial risk prediction feature sequence is used as the input value of the input gate of the LSTM deep neural network model after data filtering, normalization and loss function optimization, and then the financial risk prediction for the output gate of the LSTM deep neural network model. Considering the availability of data, small and medium-sized enterprises listed in A-share companies in the Wind database are selected as sample enterprises, and evaluation indexes are constructed and detected at the same time so as to complete the experimental design of enterprise financial risk prediction in the context of big data. The prediction of enterprise financial risk is empirically analyzed using simulation analysis and statistical analysis. The results show that in the model performance analysis, the average value of ten years of data, the highest value is still the result obtained by LSTM training, 0.761, compared with other models of LSTM deep neural network in static financial risk prediction in the overall best performance. In the case study of Yibai Pharmaceutical, the minimum value of the rate of return, return on total assets, and return on assets were -10.02%, 2.56%, -20.72%, which reflects the fact that the private enterprises still have large profitability space to be mined. This study helps investors or financial institutions such as funds to find out the possible financial risk crisis of listed companies as early as possible to avoid the parties from incurring large financial losses.

Application of 3D visual perception-driven design of velvet flowers in traditional dresses

Abstract This paper firstly analyzes the specific application of velvet flowers on women’s dresses from the perspective of fashion aesthetics and with the understanding of the background and development trend of dresses. Secondly, based on 3D visual perception, we propose the optimization of the normalization method and loss function optimization, which can improve the migration speed and generation quality of velvet flower design, and then seamlessly fit it to the dress base map by Poisson fusion to realize the intelligent design of velvet flower in dresses driven by 3D visual perception. Then, the color features and attributes of the floral in the dress are studied, and the HSV color space model is used to extract the high-frequency colors and attributes of the dress. Then, the floral design is digitized, and the dress design is driven by 3D visual perception is studied and analyzed. The results show that the loss value of the dress design style migration network trained based on AdaN is the smallest, and the loss value is reduced by 0.19 compared to the dress design style migration network using IN and 0.31 compared to BN. The velvet flower design on the dress design base map can be implemented based on the foundation provided by this study.

The predictive machine learning model of a hydrated inverse vulcanized copolymer for effective mercury sequestration from wastewater

Inverse vulcanized polysulfides (IVP) are promising sulfur-enriched copolymers with unconventional properties irresistible for diverse applications like Hg2+ remediation. Nevertheless, due to their inherent hydrophobic nature, these copolymers still offer low Hg2+ uptake capacity. Herein, we reported the synthesis of IVP by reacting molten sulfur with 4-vinyl benzyl chloride, followed by their functionalization using N-methyl D-glucamine (NMDG) to increase the hydration of the developed IVP. The chemical composition and structure of the functionalized IVP were proposed based on FTIR and XPS analysis. The functionalized IVP demonstrated a high mercury adsorption capacity of 608 mg/g (compared to <26 mg/g for common IVP) because of rich sulfur and hydrophilic regions. NMDG functionalized IVP removed 100 % Hg2+ from a low feed concentration (10–50 mg/l). A predictive machine learning model was also developed to predict the amount of mercury removed (%) using GPR, ANN, Decision Tree, and SVM algorithms. Hyperparameter and loss function optimization was also carried out to reduce the prediction error. The optimized GPR algorithm demonstrated high R2 (0.99 (training) and 0.98 (unseen)) and low RMSE (2.74 (training) and 2.53 (unseen)) values indicating its goodness in predicting the amount of mercury removed. The produced functionalized IVP can be regenerated and reused with constant Hg2+ uptake capacity. Sulfur is the waste of the petrochemical industry and is abundantly available, making the functionalized IVP a sustainable and cheap adsorbent that can be produced for high-volume Hg2+ remediation. Environmental implicationThis research effectively addresses the removal of the global top-priority neurotoxic pollutant mercury, which is toxic even at low concentrations. We attempted to remove the Hg2+ utilizing an inexpensive adsorbent developed by NMDG functionalized copolymer of molten sulfur and VBC. A predictive machine learning model was also formulated to predict the amount of mercury removal from wastewater with only a 0.05 % error which shows the goodness of the developed model. This work is critical in utilizing this low-cost adsorbent and demonstrates its potential for large-scale industrial application.

TSW-YOLO-v8n: Optimization of detection algorithms for surface defects on sawn timber

The goal of this work was to better meet the demand for rapid detection of surface defects in sawn timber in forestry production. This paper introduces a two-way feature fusion network based on the YOLO-v8 algorithm and proposes a feature fusion network model that combines the attention mechanism and loss function optimization. In this way it increases the tiny target detection head in order to more effectively detect small defective targets in the wood, thus realizing the model’s high-efficiency and low-consumption functional design. The results show that the improved TSW-YOLO-v8n model realized the identification of eight kinds of defects in sawn timber with a high efficiency of 91.10% mAP50 and an average detection 6 ms, which is 5.1% higher than the original model’s mAP50 and 1 ms shorter than the original model’s average detection time. The comparison of the original model and its mainstream algorithms shows that the model of this paper had better performance and better detection capability. Thus, the improved model achieved better overall performance and stronger detection ability, which provides a new idea for the development of detection technology in the forestry industry.

Research on the reform of university education pedagogy based on deep learning model in the context of information era

Abstract In this paper, the minimization loss function optimization deep learning algorithm is used to achieve educational pedagogical reform in the context of the information technology era by continuously optimizing the loss function to reach local minima. A feature weight threshold is used to calculate the weights of each output feature, and a weighted average pooling is performed for each one-dimensional vector to produce multiple lengths of nearest neighbor samples. The deep learning model increased the scoring frequency of key values in the reform of higher education pedagogy by about 0.18, with an accuracy rate of 0.91 and an increase of lecture board words by more than 95%. To promote the smooth promotion of college education curriculum reform, the deep learning model recommends promoting the reform of college education pedagogy.

A Deep Learning-Based Unbalanced Force Identification of the Hypergravity Centrifuge.

Accurate and quantitative identification of unbalanced force during operation is of utmost importance to reduce the impact of unbalanced force on a hypergravity centrifuge, guarantee the safe operation of a unit, and improve the accuracy of a hypergravity model test. Therefore, this paper proposes a deep learning-based unbalanced force identification model, then establishes a feature fusion framework incorporating the Residual Network (ResNet) with meaningful handcrafted features in this model, followed by loss function optimization for the imbalanced dataset. Finally, after an artificially added, unbalanced mass was used to build a shaft oscillation dataset based on the ZJU-400 hypergravity centrifuge, we used this dataset to train the unbalanced force identification model. The analysis showed that the proposed identification model performed considerably better than other benchmark models based on accuracy and stability, reducing the mean absolute error (MAE) by 15% to 51% and the root mean square error (RMSE) by 22% to 55% in the test dataset. Simultaneously, the proposed method showed high accuracy and strong stability in continuous identification during the speed-up process, surpassing the current traditional method by 75% in the MAE and by 85% in the median error, which provided guidance for counterweight and guaranteed the unit's stability.

Slender Flexible Object Segmentation Based on Object Correlation Module and Loss Function Optimization

Slender flexible objects are ubiquitous in real-world circumstances. The existing object detection and segmentation algorithms have achieved high accuracy and speed in rigid objects, but the detection effect of slender flexible objects is not ideal. Extreme aspect ratios and dynamically changeable geometric appearance characterize slender flexible objects, to the extent that it is difficult to locate them in instance segmentation. In this paper, a new instance segmentation method based on the object correlation module and loss function optimization is proposed for the detection of slender flexible objects. In order to achieve more accurate anchor box positioning, a GIoU bounding-box regression loss function is selected to overcome the problem of inconsistency between training objectives and assessment indicators. Furthermore, due to the Mask Scoring R-CNN detection network ignoring the relationships between objects, the object correlation module is proposed to achieve end-to-end learning and modeling of the correlation features between all objects in the image to improve slender flexible objects detection accuracy. The results of the experiments on the self-built flexible object dataset for the power grid operation sites demonstrate that the method presented in this research can efficiently recognize and segment slender flexible objects, with a detection accuracy of 44.8%. The ablation experiment also shows that the addition object correlation module and the revised bounding-box regression loss function are both effective and can enhance slender flexible object detection accuracy by 1.2% and 0.5%, respectively. The proposed instance segmentation method considers the correlation characteristics between objects and improves the bounding-box regression loss function to increase the segmentation accuracy of slender flexible objects.

A Client Selection Method Based on Loss Function Optimization for Federated Learning

Federated learning is a distributed machine learning method that can solve the increasingly serious problem of data islands and user data privacy, as it allows training data to be kept locally and not shared with other users. It trains a global model by aggregating locally-computed models of clients rather than their raw data. However, the divergence of local models caused by data heterogeneity of different clients may lead to slow convergence of the global model. For this problem, we focus on the client selection with federated learning, which can affect the convergence performance of the global model with the selected local models. We propose FedChoice, a client selection method based on loss function optimization, to select appropriate local models to improve the convergence of the global model. It firstly sets selected probability for clients with the value of loss function, and the client with high loss will be set higher selected probability, which can make them more likely to participate in training. Then, it introduces a local control vector and a global control vector to predict the local gradient direction and global gradient direction, respectively, and calculates the gradient correction vector to correct the gradient direction to reduce the cumulative deviation of the local gradient caused by the Non-IID data. We make experiments to verify the validity of FedChoice on CIFAR-10, CINIC-10, MNIST, EMNITS, and FEMNIST datasets, and the results show that the convergence of FedChoice is significantly improved, compared with FedAvg, FedProx, and FedNova.

Evaluation of classification ability of logistic regression model on SERS data of miRNAs.

Logistic regression (LR) is a supervised multiple linear regression model, which uses linear weighted calculation for input to obtain weight coefficients of model. The surface enhanced Raman spectroscopy (SERS) technology greatly enhances the Raman signal of analyte. LR model was used to analyze the data of seven types of pancreatic cancer-related miRNAs obtained from commercial SERS substrate. The classification ability of the model on such data was observed under the configurations of different key parameters (classification mode, regularization method and loss function optimization way), and the effect of the two types of data formats were also evaluated. The results showed that though LR model used to classify this data did not perform well as expected, miRNA-191 and miRNA-4306 still had high recalls (sensitivity), which laid a theoretical foundation for the purpose of using LR model to identify these two miRNAs to jointly diagnose of pancreatic cancer at miRNA level.

Optimasi formula effervescent berbasis nanoenkapsulat Eucalyptus citriodora

Eucalyptol (1,8-cineole) compound contained in eucalyptus oil has the potential to inhibit viral infection pathways, as an antimicrobial and as an immunoregulator. Therefore, the development of eucalyptus oil-based effervescent health drinks is urgently needed to relieve the symptoms of being infected a virus during this pandemic. However, the volatility of eucalyptus oil is quite high, so a nano-based encapsulation method is needed to slow release of these volatile compounds. This study aims to obtain the best formula for Eucalyptus citriodora oil-based effervescent granules with controlled release system in nanostructure. The experimental design used was the Taguchi method with 3 replications. The combination of level factors in the optimal formula obtained through the Taguchi loss function optimization method were effervescent materials factor-level 2, lactose/sucralose ratio factor-level 3 and nanoencapsulated/ascorbic acid ratio factor-level 2. The optimal formula was (1) effervescent materials factor: 24% sodium bicarbonate, 12% tartaric acid, 11% citric acid; (2) lactose/sucralose ratio factor: 46.2% / 0.2%; (3) nanoencapsulated/ascorbic acid ratio factor: 4.5% / 2%. The optimal product characteristics were water content of 5.52%, pH of 5.9, dissolving time of 106 seconds, carr's index of 23.56, hausner's ratio of 1.31, and total essential oil content of 0.014mL/4gram. This product can be used as a health drinks (throat lozenges) and immune booster.

Surface Defect Detection with Modified Real-Time Detector YOLOv3

In this paper, a modified YOLOv3 net has been proposed for surface defect detection. Different from other pixel-level segmenting methods, YOLOv3 locates the regions of surface defects with bounding rectangles. Compared with conventional detectors, the operating efficiency of YOLOv3 is rather high without generating region proposals by sliding boxes. Although pixel-level details of defects are omitted in the process, the primary information of the location of detects and class labels are extracted by YOLOv3 with high accuracy. This information is sufficient for surface defect inspection, and computational efficiency has been improved, simultaneously. To further light the structure of YOLOv3, loss function optimization and pruning strategy have been adopted in the original YOLOv3. The pruning ratio is determined by the tradeoff between detecting accuracy and computational efficiency. In our experiments, we compared the performance of modified YOLOv3 with several state-of-the-art methods, and modified YOLOv3 achieves the best performance on six types of surface defects in DAGM 2007 dataset.

Fuzzy Information Discrimination Measures and Their Application to Low Dimensional Embedding Construction in the UMAP Algorithm.

Dimensionality reduction techniques are often used by researchers in order to make high dimensional data easier to interpret visually, as data visualization is only possible in low dimensional spaces. Recent research in nonlinear dimensionality reduction introduced many effective algorithms, including t-distributed stochastic neighbor embedding (t-SNE), uniform manifold approximation and projection (UMAP), dimensionality reduction technique based on triplet constraints (TriMAP), and pairwise controlled manifold approximation (PaCMAP), aimed to preserve both the local and global structure of high dimensional data while reducing the dimensionality. The UMAP algorithm has found its application in bioinformatics, genetics, genomics, and has been widely used to improve the accuracy of other machine learning algorithms. In this research, we compare the performance of different fuzzy information discrimination measures used as loss functions in the UMAP algorithm while constructing low dimensional embeddings. In order to achieve this, we derive the gradients of the considered losses analytically and employ the Adam algorithm during the loss function optimization process. From the conducted experimental studies we conclude that the use of either the logarithmic fuzzy cross entropy loss without reduced repulsion or the symmetric logarithmic fuzzy cross entropy loss with sufficiently large neighbor count leads to better global structure preservation of the original multidimensional data when compared to the loss function used in the original UMAP algorithm implementation.

Short-term multi-energy load forecasting for integrated energy systems based on CNN-BiGRU optimized by attention mechanism

Accurate short-term multi-energy load forecasting is an essential prerequisite for ensuring the reliable and economic operation of integrated energy systems (IES). Considering the large fluctuations, strong randomness, and the multi-energy coupling relationship of regional IES, this paper proposes a novel short-term multi-energy load forecasting method based on a CNN-BiGRU model that is optimized by attention mechanism. First, the dynamic coupling relationship between multi-energy loads is qualitatively analyzed, and the influencing factors of multi-energ loads are screened based on data-driven analysis. Second, a one-dimensional CNN layer is formulated to extract complex high-dimensional features, and BiGRU is constructed to extract the time dependence from historical sequences. In particular, three attention mechanism modules are introduced to the BiGRU hidden state through the mapping weight and learning parameter matrix to enhance the impact of key information. Then, hard weight sharing is adopted to extract the inherent multi-energy coupling relationship. Finally, a novel multi-task loss function weight optimization method is applied to search for the optimal multi-task weight, which is used to balance multi-task learning (MTL) to achieve the optimization of the overall forecasting model. To validate the effectiveness of the CNN-BiGRU-Attention MTL model with loss function optimization, this paper compares the proposed model with five benchmark models by MAPE, RMSE, MAE, R2, and computational time. Compared with the traditional LSTM model, the cooling, heat, and electrical load forecasting accuracy (measured by MAPE) of the proposed hybrid model increased by 61.86%, 73.03%, and 63.39%, respectively, which demonstrates that the proposed model exhibits superior performance.

Performance enhancement‐based active learning sample selection method

AbstractRepresentative samples are important for multivariate calibration. The highly efficient selection of representative samples to be labelled can save money and time. Existing methods, such as Kennard‐Stone and net analyte signal selection, are usually based on the distance between candidate samples and labelled calibration sets in feature space. However, these distances are influenced by the feature space, which is spanned by an information vector extracted from labelled samples. To overcome the negative effects of the distance‐based selection method, a model performance enhancement‐based sample selection method is proposed to select calibration samples efficiently. Based on loss function optimization, the samples that can improve model performance the most, as estimated by bootstrap, are sequentially selected and added to the calibration set. Due to the high representation of each sample, a few samples can build a model that has no significant loss of prediction ability when compared with a model built with the large number set of calibration samples. The performance enhancement‐based active learning (PEAL) sample selection method is both effective and efficient.

BroadGAN: Generative adversarial networks of discriminating separate features based on broad learning

Since the generative adversarial network (GAN) was proposed in 2014, it has rapidly become a hot topic in the field of deep learning. In recent years, there are many optimizations for GAN, which are divided into three kinds, including the optimization of loss functions, external structure of network and internal structure of network. Few people optimizes GAN from internal structure of network, and because of the process of game, slow training speed is one of the biggest problems of GAN. This paper introduces the ideology of broad learning algorithm (Chen and Liu, 2017) to put forward the multi-judge generative adversarial network method based on different random features to enhance the quality of generative result and increase the training speed of GAN (Goodfellow et al., 2014). This model provides a method which reduce the input information of each discriminator to optimize the training process of GAN. The experiments on cifar10 and anime face dataset find that our model obtains a better performance than the GMAN model and Base model. This paper finds a group of hyper-parameters to enhance the BroadGAN.